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ABSTRACT 

The  Hazard  Prediction  and  Assessment  Capability  (HP AC)  is  a  hazard  modelling  tool  that 
predicts  the  effects  of  the  release  of  toxic  materials  to  the  atmosphere.  Detailed  meteorological 
data  is  required  to  obtain  reliable  modelling  results,  including  surface  observations,  upper-air 
profiles  and  gridded  forecast  data.  HPAC  is  a  US  developed  model  and  cannot  interpret 
locally  available  meteorological  data.  This  report  describes  the  DSTO-built  computer 
program  AWSMSTR  which  retrieves  the  relevant  data  from  a  remote  server  and  reformats  it 
for  use  in  HPAC. 
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Reformatting  Meteorological  Data  for  use  in  the 
Hazard  Prediction  and  Assessment  Capability 


Executive  Summary 

The  Hazard  Prediction  and  Assessment  Capability  (HP AC)  is  used  by  the  Incident 
Response  Regiment  (IRR)  to  predict  the  effects  of  the  release  of  toxic  materials  to  the 
atmosphere.  The  reliability  of  these  predictions  is  heavily  reliant  on  the  input  data, 
especially  the  meteorological  data,  which  is  used  to  build  the  wind  field  and 
turbulence  structure  that  drives  the  dispersion  calculation.  Depending  on  the  scenario 
being  modelled  this  meteorological  data  may  be  in  the  form  of  surface  and  upper-air 
observations  or  forecast  data  from  mesoscale  model  runs. 

In  Australia,  this  meteorological  data  is  produced  by  the  Bureau  of  Meteorology 
(BoM).  HPAC  was  developed  in  the  United  States  and  so  is  not  compatible  with 
Australian  meteorological  data  formats.  This  report  describes  work  undertaken  by 
DSTO  to  construct  a  computer  program  (called  AWSMSTR)  that  automatically 
downloads  requested  data  and  reformats  it  for  use  in  HPAC.  Included  in  the  report 
are  descriptions  of  both  the  BoM  and  HPAC  formats  for  various  data  types  and  the 
formulae  used  in  the  conversions.  Justification  for  the  assumptions  made  when 
converting  gridded  forecast  data  and  an  explanation  of  the  probable  errors  involved 
are  also  provided. 

The  AWSMSTR  software  enables  Australian  HPAC  users  in  both  the  defence  and  civil 
communities  to  access  the  most  up-to-date  meteorological  information  available.  The 
software  has  been  designed  to  ensure  that  the  reformatting  of  new  data  formats  is  as 
simple  as  possible  and  is  available  to  licensed  HPAC  users  upon  request  to  the  authors. 
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1.  Introduction 


The  suite  of  modelling  tools  known  as  HPAC  (Hazard  Prediction  and  Assessment 
Capability)  is  used  by  the  Incident  Response  Regiment  (IRR)  to  predict  the  hazards  that 
result  from  releases  of  Chemical,  Biological,  Radiological  and  Nuclear  (CBRN) 
materials  into  the  atmosphere.  HPAC,  developed  in  the  US  by  the  Defense  Threat 
Reduction  Agency  (DTRA)  calculates  the  initial  cloud  of  material  resulting  from  the 
use  of  Weapons  of  Mass  Destruction  and  the  resulting  dispersion.  This  information  is 
provided  to  the  user  of  the  software  in  the  form  of  plots  of  concentrations,  doses, 
deposited  material,  etc. 

The  dispersion  process  is  heavily  reliant  on  the  meteorological  conditions  present.  To 
be  used  effectively,  HPAC  requires  not  only  surface  and  upper-air  observations,  but 
also  detailed  gridded  forecast  data.  This  data  is  readily  available  in  the  United  States, 
but  has  not  been  available  in  Australia  in  a  format  that  can  be  interpreted  by  HPAC. 
The  Bureau  of  Meteorology  (BoM)  collects  large  amounts  of  observational  data  from 
across  the  country  and  uses  Numerical  Weather  Prediction  (NWP)  models  to  produce 
detailed  forecasts. 

This  report  explains  the  process  undertaken  by  DSTO  to  create  specialised  programs 
that  automatically  download  both  observational  and  forecast  data  from  the  BoM  and 
process  it  into  formats  that  HPAC  understands.  It  also  provides  some  detail  on  the 
formats  themselves,  including  the  HPAC  internal  formats  and  how  they  relate  to  the 
standards  used  by  the  BoM. 


2.  Observational  Data  Formatting 

2.1  Surface  Observations 

HPAC  can  read  surface  observations  from  weather  files  in  its  native  format,  known  as 
".sfc",  and  also  has  the  ability  to  import  international  METAR  (Meteorological 
Aerodrome  Report)  standard  format  files. 

Surface  observations  from  the  BoM  are  in  ".axf"  format,  its  standard  for  surface 
observations  (axf  stands  for  AIFS  (Australian  Integrated  Forecast  System)  Exchange 
Format).  The  .axf  format  is  used  to  record  data  from  various  observational  stations, 
such  as  automatic  reporting  stations,  manned  stations  and  harbour  buoys,  etc.  The 
data  from  these  stations  are  recorded  in  different  types  of  .axf  format,  depending  on 
the  end-user  and  the  nature  of  the  data.  The  two  main  types  of  .axf  format  that  were 
considered  by  DSTO  for  use  with  HPAC  are  "[SURFACE]"  and  "[metar2Data]". 
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2.1.1  BoM  format 

During  the  visit  to  the  BoM  in  mid  2003,  DSTO  was  provided  with  an  example  of  the 
[SURFACE]  .  axf  type  for  recording  surface  observations.  Due  to  the  similarity  of  this 
format  to  the  native  HPAC  surface  observation  format,  .sfc,  the  reprocessing  of 
[SURFACE]  files  was  initially  considered  to  be  the  best  option. 

However,  most  HPAC  projects  will  only  require  a  small  proportion  of  the  total 
available  surface  observations  over  Australia.  It  was  eventually  decided  that  the 
METAR  format  would  be  preferable  for  surface  observations  as  HPAC  allows  an 
efficient  filtering  of  irrelevant  observations.  There  is  some  loss  in  accuracy  in  the 
importing  process  due  to  rounding  errors,  as  METAR  can  only  deal  with  integral 
values  of  the  temperature  and  dewpoint.  To  minimise  reformatting,  the  BoM  supplied 
the  surface  observations  in  another  .axf  type,  called  [metar2Data],  details  of  which 
are  presented  in  Table  1. 


Table  1:  Detail  of  [metarlData]  aviation  .axf  type  format 


BoM  Variables 

ID_num 

ID_name 

Date 

Time 

Lat 

Variable 

Description 

5  Digit  Station 
Identifier 

4  Letter  Station 
Identifier 

Date 

(YYYYMMDD) 

Time  (hhmm) 

Latitude  (deg  N) 

Example  Values 

95304 

YBWX 

20031008 

0448 

-20.88 

BoM  Variables 

Lon 

Wdir 

Wspd 

T_DB 

DP 

Variable 

Description 

Longitude  (deg 

E) 

Wind  Direction 
(deg) 

Wind  Speed 
(knots) 

Temperature 

Dry  Bulb  (°C) 

Dewpoint  Temp 
(°C) 

Example  Values 

115.41 

050 

15 

30.7 

11.7 

BoM  Variables 

QNH 

RF9am 

RFIOm 

Vis 

Avis 

Variable 

Description 

QNH  Pressure 
(hPa) 

Precipitation 
Since  9  am 
Local  fmmi 

Precipitation  - 
Last  10  mins 
(mm) 

Not  used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

1013.5 

0.0 

0.0 

-9999 

-9999 

BoM  Variables 

Gust 

Wxllnt 

WxIDsc 

WxlWxl 

Wx1Wx2 

Variable 

Description 

Max  Wind  Gust 
Since  9am  (kts) 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

20 

-9999 

-9999 

-9999 

-9999 

BoM  Variables 

Wx1Wx3 

Wx2lnt 

Wx2Dsc 

Wx2Wx1 

Wx2Wx2 

Variable 

Description 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

-9999 

-9999 

-9999 

-9999 

-9999 

BoM  Variables 

Wx2Wx3 

CtdIAmt 

CldITyp 

CldIBase 

Cld2Amt 

Variable 

Description 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

-9999 

-9999 

-9999 

-9999 

-9999 

BoM  Variables 

Cld2Typ 

Cld2Base 

Cld3Amt 

Cld3Typ 

Cld3Base 

Variable 

Description 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

-9999 

-9999 

-9999 

-9999 

-9999 
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BoM  Variables 

Cld4Amt 

Cld4Typ 

Cld4Base 

Ceil  1  Amt 

Ceil  1  Base 

Variable 

Description 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

-9999 

-9999 

-9999 

-9999 

-9999 

BoM  Variables 

Ceil2Amt 

Ceil2Base 

Ceil3Amt 

Ceil3Base 

Variable 

Description 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Not  Used  by 
HPAC 

Example  Values 

-9999 

-9999 

-9999 

-9999 

Notes:  1)  If  data  is  not  available,  then  the  value  -9999  is  used  as  a  flag 

2)  Each  observation  takes  up  a  single  "line"  in  the  observation  file  (comma 
separated)  -  these  lines  when  concatenated  form  the  observation  file  (Appendix  A) 

3)  Data  not  imported  by  HP  AC  includes  precipitation,  visibility  and  other  more 
general  cloud  information  and  remarks 


The  [metar2Data]  type  contains  a  lot  of  variables  whose  values  are  not  used  in  HPAC, 
creating  an  inefficiency  that  would  normally  be  avoided.  However,  due  to  the  very 
small  file  sizes  and  the  effort  that  would  be  required  by  the  BoM  to  deliver  a  non¬ 
standard  format,  this  has  not  arisen  as  an  issue. 

Some  surface  weather  stations  report  every  15  minutes,  while  others  have  a  reporting 
interval  of  up  to  three  hours.  This  means  that  when  selecting  any  given  set  of 
observation  files,  the  latest  observation  from  all  stations  within  the  modelling  domain 
may  not  be  included.  A  very  simple  way  to  ensure  that  up  to  date  observations  are 
used  is  to  select  a  set  of  weather  files  spanning  at  least  three  hours  before  until  three 
hours  after  the  boundaries  of  the  model's  temporal  domain  when  importing  surface 
observations. 

2.1.2  HPAC  Format 

The  HPAC  User's  Guide  [1]  contains  a  detailed  description  of  the  HPAC  native  .  sf  c 
format.  It  also  contains  some  information  on  the  standard  METAR  format,  a  more 
rigorous  definition  of  which  can  be  found  in  the  meteorology  text  [2]  by  Whiteman. 
There  are  many  variables  in  the  standard  METAR  format  that  are  not  utilised  by 
HPAC,  so  the  final  format  used  for  the  surface  observations  is  not  strictly  a  standard 
METAR  format.  Table  2  below  gives  a  brief  description  of  the  variables  that  are 
included  in  the  HPAC  METAR  files.  An  example  output  file  can  be  found  in  Appendix 
A. 

The  data  is  saved  in  an  ASCII  text  file  with  the  extension  " .  sao"  in  space  delimited 
format,  with  each  observation  beginning  on  a  new  line,  as  required  by  HPAC.  The 
downloaded  files  vary  in  size  as  the  automatic  weather  stations  report  at  a  range  of 
frequencies,  with  the  largest  files  being  around  125  kb.  The  reformatting  process 
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removes  a  lot  of  the  information,  so  the  METAR  files  that  are  output  from  the  program 
are  typically  reduced  in  size  by  a  factor  of  approximately  seven. 


Table  2:  METAR  format  used  in  HP  AC 


Variable  Form 

Obs_Type 

LLLL 

DDhhmmZ 

DDDssKT 

//// 

DB/DP 

Qpppp 

Variable 

Description 

Label  indicating 
Meteorological 
Aerodrome 
Report 

4  Letter 
identifier  for 
station 

Date  &  Time, 
with  Z  appended 
indicating  UTC 

Wind  direction 
&  speed,  with 
KT  indicating 
knots 

Slashes 
indicate 
missing  or 
unused  data 

Dry  Bulb  Temp 
and  Dewpoint 
separated  by  / 

Pressure 
(hPa)  (Q 
indicates 
QNH) 

Example 

Value 

METAR 

YHLC 

161230Z 

15009KT 

//// 

24/20 

Q1012 

Notes:  1)  Tlte  4  central  slashes  -////-  indicate  that  there  are  no  comments  for  this 

report.  HP  AC  ignores  these  comments  if  they  exist. 

2)  If  temperature  or  dewpoint  are  less  than  zero,  then  they  are  prefixed  with 
the  letter  M  to  indicate  negative  temperatures  (i.e.  -12  is  M12). 

3)  If  data  is  not  available  for  any  variable,  it  is  replaced  by  ////,  except  in  the 
case  of  pressure,  which  becomes  QJ///. 

4)  Data  not  imported  by  HPAC  includes  precipitation,  visibility  and  other 
more  general  cloud  information  and  remarks. 

Each  observation  records  only  the  four-letter  ID  code  as  a  means  of  identifying  the 
station  (also  known  as  the  International  Civilian  Aviation  Organisation  (ICAO) 
identifier),  but  HPAC  also  requires  the  location  of  this  station  to  make  use  of  the  data. 
This  is  done  by  a  simple  process  of  using  the  four-letter  ID  code  as  a  key  to  a  lookup 
table,  and  HPAC  stores  this  information  in  the  file  wmo.stn,  which  is  stored  in  the 
C:\HPAC4\client\runs\data\wxstn  and  C:\HPAC4\client\data\wxstn 
directories.  The  format  of  this  look-up  table  is  explained  in  section  7.6.2  of  the  User's 
Guide  [1].  The  BoM  provided  DSTO  with  a  list  of  all  stations  and  their  corresponding 
ID  codes,  which  was  used  to  update  the  wmo .  stn  file.  Note  that  the  upper  air  format 
uses  a  similar  system,  but  identifies  stations  via  their  5  digit  ID's. 

Importing  the  METAR  surface  observation  files  into  HPAC  then  proceeds  via  the 
Weather  Wizard,  Import  Other  Files  >  Open  >  Scattered  >  Select  Max  Stations  >  OK. 

2.2  Upper  Air  Observations 

The  upper-air  observations  supplied  by  the  BoM  are  in  the  standard  RAOBS 
(Rawinsonde  Observation)  format,  which  HPAC  can  import  easily.  The  RAOBS  format 
is  quite  complicated,  but  an  overview  can  be  found  in  Appendix  C  of  the  User's  Guide 
[1].  The  Federal  Meteorological  Handbook  [3]  contains  much  more  detailed 
information  on  the  RAOBS  format  and  is  available  on  the  Internet.  The  data  are 
written  in  ASCII  format,  and  so  can  be  viewed  by  most  text  editors.  The  only 
modification  that  HPAC  requires  is  that  the  file  extension  be  changed  from  .  txt  to 
.  wmo.  The  process  to  load  the  files  into  HPAC  is  the  same  as  for  the  surface  METARs, 


4 


DSTO-TN-0595 


and  users  will  normally  import  both  upper  air  and  surface  observations 
simultaneously  by  selecting  files  with  extensions  .sao  and  .wmo.  HP  AC  will  extract 
all  observations  from  within  its  modelling  domain  and  write  them  to  an  .sfc  and 
.prf  file,  where  .prf  is  the  native  HP  AC  format  for  upper  air  observations  (see 
below). 

HP  AC  can  process  the  different  parts  of  the  RAOBS  coded  observation,  TTAA 
(mandatory  levels),  TTBB  (significant  levels  with  respect  to  temperature/humidity), 
PPBB  (significant  levels  with  respect  to  wind)  and  the  three  levels  that  provide  100  hPa 
to  10  hPa  data,  TTCC,  TTDD  and  PPDD.  These  files  are  very  small,  usually  around  1- 
30  kb,  so  they  are  processed  very  quickly. 


3.  Forecast  Data  Reformatting 

3.1  NetCDF  Format 

The  BoM  produce  NWP  gridded  forecasts  using  the  Mesoscale  Limited  Area  Prediction 
System  (MesoLAPS)  every  12  hours.  This  data  is  written  in  NetCDF  (Network 
Common  Data  Format)  format,  commonly  used  for  scientific  data.  Unfortunately 
HP  AC  cannot  interpret  NetCDF  data,  so  a  major  part  of  the  project  has  been  to  write  a 
conversion  program  that  translates  from  NetCDF  to  HPAC's  format  for  gridded  data. 
The  MesoLAPS  NetCDF  files  supplied  by  the  Bureau  of  Meteorology  are  stored  in  a 
binary  form  and  contain  the  variables  shown  below  in  Table  3. 


Table  3:  Current  NetCDF  format  variable  definitions 


Variable  Name 

Description 

Units 

Varies  with 

Used  in 
reformatting 

Ion 

Longitudes 

degrees  E 

1  -D  Array 

Yes 

lat 

Latitudes 

degrees  N 

1  -D  Array 

Yes 

Ivl 

Vertical  Levels 

1  -D  Array 

Yes 

time 

Days  Since  Forecast 

time 

scalar 

No 

seg_type 

Header  Information 

char_size,  time 

No 

tm_step_size 

Time  Step  Size 

seconds 

scalar 

No 

basedate 

Base  Date  of  Data  Segment 

date 

string 

No 

base_time 

Base  Time  of  Data  Segment 

time 

string 

No 

valid_date 

Valid  Date  of  Data  Segment 

date 

string 

Yes 

valid_time 

Valid  Time  of  Data  Segment 

time 

string 

Yes 

air_temp 

Air  Temperature 

K 

time,  Ivl,  lat,  Ion 

Yes 

zonal_wnd 

U  Direction  (W-E)  Winds 

m/s 

time,  Ivl,  lat,  Ion 

Yes 

merid_wnd 

V  Direction  (N-S)  Winds 

m/s 

time,  Ivl,  lat,  Ion 

Yes 

omega 

W  Direction  (vertical)  Winds 

Pa/s 

time,  Ivl,  lat,  Ion 

Yes 

mix_rto 

Humidity  Ratio 

kg/kg 

time,  Ivl,  lat,  Ion 

Yes 

sfc  pres 

Surface  Pressure 

Pa 

time,  lat,  Ion 

Yes 
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geop_ht 

Geopotential  Height 

m 

time,  Ivl,  lat,  Ion 

Yes 

mslp 

Mean  Sea  Level  Pressure 

hPa 

time,  lat,  Ion 

No 

topog 

Terrain  Elevation 

Atmospheric  Boundary-Layer 

m 

time,  lat,  Ion 

Yes 

abl_ht 

Height 

m 

time,  lat,  Ion 

Yes 

sfc_ttl_heat_flx 

Total  Surface  Heat  Flux 

W/m2 

time,  lat,  Ion 

Yes 

Note  that  initially  DSTO  received  just  the  raw  NetCDF  files  with  all  available  variables 
included,  as  produced  by  the  BoM  for  other  purposes.  However,  some  variables  that 
were  critical  to  DSTO  were  not  normally  used  by  the  BoM  and  hence  were  included 
only  as  dummy  values.  BoM  were  then  able  to  provide  a  smaller,  custom-built  set  of 
files  in  the  NetCDF  format  for  use  with  HPAC.  The  NetCDF  files  are  in  a  binary 
format,  so  a  NetCDF  viewer  is  required  to  access  the  data  and  labels  -  several  of  these 
are  available  on  the  Internet  [4]. 

The  resolution  of  the  data  is  reasonable,  with  a  5  km  grid  size  for  the  Sydney  and 
Melbourne  regions,  and  12  km  gridded  data  available  for  all  other  major  centres.  To 
obtain  reliable  hazard  estimates  when  modelling  over  small  spatial  domains  requires 
higher  resolution  data,  but  this  is  not  currently  available  in  Australia. 

3.2  Gridded  Weather  Formats  Accepted  by  HPAC 

3.2.1  HASCAL  Gridded  Format 

HPAC  has  its  own  native  gridded  weather  format,  known  as  the  HASCAL  (or  HPAC) 
gridded  format.  The  HASCAL  grid  has  no  flexibility  in  the  types  of  data  that  can  be 
included;  it  demands  that  the  only  four  inputs  are  the  U  (east-west)  and  V  (north- 
south)  components  of  wind  velocity,  temperature  and  relative  humidity.  The  other 
option  for  inputting  gridded  forecast  data  into  HPAC  is  the  MEDOC  format  (described 
in  detail  below),  which  allows  for  many  different  combinations  of  variables,  and  so  was 
preferred  over  the  HASCAL  format.  More  detail  on  the  HASCAL  format  can  be  found 
in  the  User's  Guide  [1]  and  the  SCIPUFF  Technical  Document  [5], 

3.2.2  Upper  Air  Profile  Format 

Another  option  that  was  explored  was  to  use  the  internal  HPAC  format  for  upper  air 
profiles  (.prf  files)  to  write  the  grid.  This  was  examined  because  the  NetCDF  format 
is  written  in  terms  of  constant  pressure  levels,  whilst  MEDOC  demands  constant 
height  levels.  The  .prf  format  also  uses  pressure  levels,  so  this  would  save 
considerable  programming  and  reduce  errors  due  to  interpolating  data  from  pressure 
levels  to  height  levels.  Conversely,  HPAC  ingests  gridded  data  much  more  efficiently 
than  profile  data,  and  run  times  for  profiles  would  be  significantly  higher  when 
compared  with  the  corresponding  MEDOC  files.  Since  model  run  times  are  likely  to  be 
critical  when  using  HPAC  operationally  it  was  decided  that  the  .prf  format  was  not 
appropriate. 
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3.2.3  MEDOC  Format 

The  MEDOC  format  was  developed  for  use  by  the  French  Electricity  Board  in  pollutant 
dispersal  programs.  The  developers  of  HPAC,  the  DTRA  (then  the  Defense  Nuclear 
Agency)  were  involved  with  this  work,  hence  the  connection  between  MEDOC  and 
HPAC.  MEDOC  was  not  specifically  designed  for  HPAC,  so  there  are  aspects  of  the 
format  that  are  ignored,  such  as  visibility  information  and  precipitation  rates.  The 
HPAC  Users  Guide  [1]  and  the  SCIPUFF  Technical  Document  [5]  contain  details  of  the 
MEDOC  format,  but  are  brief  and  at  times  contradictory,  so  some  aspects  will  be 
elaborated  upon  here. 


Table  4:  The  MEDOC  format  description 


8  letter  string,  F’s  for  formatted,  B's  for  binary 

FFFFFFFF 

Name  of  the  code  used  (NOT  used  by  HPAC) 

TRNSGRIB 

- - Bay  Month  Year 

~ . 12 - - - Wk 


"Tear" 


Time  of  this  data  set 


Minute 

_ _ U _ 

Minutfe" 

0 


"Second-,, 


- - 

-Second.,, 


Time  of  initial  calculation 
(NOT  used  by  HPAC) 


Set  special  points  to  zero 


,,--rofGrid 

Points  -  x 

#  of  Grid  #  of  Grid-N  Points  (NOT 

Points  -  y  Points  •  z  )  Used  by  HPACf 

^/Number  3D 
variables 

Number  2D 
variables 

—  4 

3 

X'b"'''  0  M 

6 

l 

. \ 

Hence  number  of  grid  points 
=  4x3x15  =  180 

Dummy  variable 

Dummy 

variable 

Dummy  variable  Dummy  variable  Dummy  variable  Dummy  variable 

0 

0 

0  0 

0 

0 

Dummy  variable 

0 

Terrain  following  grid 
levels 


Dummy 

variable 

0 


Dummy  variable 

0 


Set  dummy  variables  to  zero, 
iahle  N  ignored  by  HPAC 


The  height  above  sea  level  of  the  southwest  corner  of 
the  grid  is  defined  for.  each  of  the  15  vertical  levels 


,'"145.7986 

/ 

I  1985.2294 

359.5139 

3041 . 166 

576.3943  ^  798.9402 

5611.5967  7230.8535 

1025.5447 

9199.6895 

14  94 . 74  05'> 
10379 . 626,1 

i 

i 

i 

(Grid  Spacing  (m  Grid  Spacing  (m 

Grid  Origin  -  x 

i 

t 

\  or  deg) 

or  deg) 

(kms) 

\11766 .4102 

13523 . 1162 

16  04  7.814  S.''  1 

l 

-999999 

Grid  Origin 

Grid  Origin  -  y  (kms) 

flat) 

Grid  Origin  (Ion)  Dummy  variable  Dummy  variable  Dummy  variable 

-999999 

39.43 

-113.7  0 

0 

0 
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Names  of  2D 
variables 

TOPO 


Dummy  variable  Vertical  Domain  height  (m) 

n  0 


Name  of  3D  variables 


If  Vertical  domain  height  not  specified  (zero 
value),  then  max  vertical  height  for  meteorology 
is  assumed  to  be  HPAC  project  domain  height 


u 

V 

w 

T 

H 

PHI 

Units  of  3D  variables 

M/S 

M/S 

M/S 

KELVIN 

GM/GM 

METERS 

Units  of  2D 
variables 

METERS 


Other  possible  variables 
include  wind  variance,  heat  flux 
and  boundary  layer  height 


4.7194 

4 . 0831 

3.123 

3 . 1447 

5.181 

3.8197 

2.6974 

3.1469 

4.77 

3.6373 

3.561 

4.6838 

U  winds  data  (west  is 
180  points,  in  m/s 

positive), 

8.6923 

10.4461 

12 . 1306 

13.4434 

8.3102 

9.6627 

11 . 1441 

12 . 349 

8 . 9767 

9.9146 

10.9582 

11 . 8431 

List  of  data  for  second  3D  variable,  read  from  left  to  right  (W  to  E),  then  up  (S  to  N),  then  vertically 


-2.9386 

-1.6359 

-0.0655  0.8006  -1.9595  -0.1594 

0.6897 

0.3719 

-1.7097  0.3943  0.8394  -0.1985 

V  winds  data  (north  is  positive), 
180  points,  in  m/s 

-2 . 987 

-3 .2447 

-3.7645  -4.1708  -4.6217  -4.9667 

List  of  data  for  third  3D  variable,  read  from  left  to  right  (W  to  E),  then  up  (S  to  N),  then  vertically 

0 

0 

0  0  0 

0 

0 

0 

0  0  0 

0 

.  .  . 

.  .  . 

W  winds  data  (vertical  winds),  180 
points,  in  m/s 

.  .  . 

0 

0 

0  0  0 

0 

286.3005 

285.793 

286.2977 

285.7087 

285.9251  285.467 

284.5406  285.2825 

285.0277 

285.6671 

285.5887 

285.6915 

::: 

Potential  temperature  (K) 
at  each  point 

412 . 1619 

413 . 1445 

411.0605  411.671 

412.4677 

413.499 

List  of  data  for  fifth  3D  variable,  read  from  left  to  right  (W  to  E),  then  up  (S  to  N),  then  vertically 


0.0023 

0.0024 


0 . 0026 
0.0028 


0.0027 

0.0024 


0.0031 

0.0023 


0.0025 

0.0024 


0.0025 

0.0026 


Humidity  ratio  (gram/gram) 
at  each  point 
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0 

0 

0  0  0 

0 

List  of  data  for  sixth  and  final  3D  variable,  read  from  left  to  right  (W  to  E),  then  up  (S  to  N),  then  vertically 

154.1142 

135.0083 

145.0238 

120 . 0095 

132.6476  120.1434  160.2519 

159.3593  147.2832  131.7708 

149.8387 

116.0533 

PHI  -  geopotential  height  (m)  -  no 
longer  used  by  HPAC 

16005.5049 

16002 . 068 

16003.5996  15997.9414  15992.5869 

15987 . 

5498 

List  of  data  for  single 

i 2D  variable,  read  from  left  to  right  (W  to  E),  then  up  (S  to  N) 

2078.2341 

2142.7893 

2103.6545 

2338 . 1506 

2229.3638  2272.1587  1875.5222 

1740.0813  1796.8811  2076.2375 

1904.1626 

2346.7925 

Process  then  repeats  for  each  time-step  included  in  the  MEDOC  file. 

The  MEDOC  file  has  three  main  sections  -  the  header  section,  which  contains 
information  about  the  grid  and  the  variables  that  are  included  in  the  file,  the  3-D 
variables  section,  which  includes  wind,  temperature,  etc.  and  their  values,  and  the  2-D 
variables  section,  which  describes  topography  and  heat  flux.  Table  4  contains  a  basic 
description  of  a  MEDOC  file  with  example  values  (shaded)  from  a  small  4x3  grid  with 
15  vertical  levels  at  one-degree  spacings  and  notes  on  how  HPAC  interprets  the 
MEDOC  data. 

3.3  Interpolation  Issues 

Unfortunately,  the  MesoLAPS  and  MEDOC  grids  that  form  the  basis  of  the  forecast 
data  are  quite  different.  The  MesoLAPS  files  record  all  data  on  the  basis  of  fixed  sigma 
levels,  which  correspond  to  constant  fractions  of  the  surface  pressure.  This  type  of 
structure  is  very  common  in  meteorological  modelling,  both  in  Australia  and  overseas. 
Unfortunately  the  MEDOC  data  is  recorded  at  levels  of  fixed  vertical  height  relative  to 
the  ground  (so  terrain  is  incorporated  into  the  file  without  the  need  to  specify  a  terrain 
file  separately).  This  incompatibility  necessitates  the  interpolation  of  all  three- 
dimensional  data  to  constant  height  levels  (defined  by  the  height  of  the  southwest 
corner  of  the  domain)  from  constant  sigma  levels  (fraction  of  the  surface  pressure). 
Figure  1  shows  a  cross-sectional  view  of  the  situation,  with  the  blue  horizontal  lines 
representing  lines  of  constant  atmospheric  pressure.  The  red  contours  represent  the 
levels  on  the  NetCDF  files,  which  are  a  fraction  of  the  pressure  at  the  surface,  and  the 
black  dotted  line  shows  the  MEDOC  level  closest  to  the  surface.  Due  to  the  non-linear 
decrease  in  pressure  with  linear  increase  in  height,  we  expect  the  vertical  difference 
between  the  MEDOC  and  NetCDF  levels  (in  metres)  to  increase  with  increasing  height, 
where  the  rate  of  change  of  pressure  with  respect  to  height  is  greatest.  This  is  clearly 
shown  in  Figure  1  where  the  greatest  distance  between  the  NetCDF  and  MEDOC  grids 
occurs  at  the  highest  point  in  the  terrain.  So  any  interpolation  carried  out  will  have 
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greater  error  at  points  of  higher  terrain,  but  note  that  the  magnitude  of  the  error  is  not 
expected  to  be  large  over  most  of  Australia. 


Using  the  ideas  from  Figure  1,  a  variable  mapping  table  was  drawn  up  linking  NetCDF 
variables  with  corresponding  MEDOC  variables,  as  shown  in  Table  5  below.  Note  that 
the  NetCDF  and  MEDOC  variables  are  in  italic  and  bold  face  respectively.  The 
position  of  the  grid  is  defined  by  the  lat  &  Ion  NetCDF  variables  which  define  the 
southwest  corner  of  the  grid.  Temporal  information  also  transfers  almost  directly  from 
NetCDF  to  MEDOC  format. 


Table  5:  NetCDF  to  MEDOC  variable  mapping 


NetCDF 

Variable 

MEDOC  Variables 

Comments  &  Adjustments 

lat  (deg_N) 

LAT  -  Single  scalar  of  southern  most  point 

DX  -  Grid  spacing  in  north-south  direction  (deg) 

IMAX  -  Number  of  grid  points  in  x  direction 

The  lat  variable  is  written  directly  to  its 
MEDOC  counterpart  LAT 
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LON  -  Single  scalar  of  western  most  point 

Ion  (deg_E) 

DY  -  Grid  spacing  in  east-west  direction  (deg) 

The  Ion  variable  is  written  directly  to  its 
MEDOC  counterpart  LON 

JMAX  -  Number  of  grid  points  in  y  direction 

ID  AY  -  dd  (day)  component  of  valid_date 

valid_date 

(yyyymmdd) 

IMONTH  -  mm  (month)  component  of  valid_date 

The  date  is  mapped  directly 

IYEAR  -  yyyy  (year)  component  of  valid_date 

IHOUR  -  hh  (hour)  component  of  valid Jime 

valid  time  (hhmm 
UTC) 

IMIN  -  mm  (minute)  component  of  validjime 

The  time  is  mapped  directly,  “seconds"  data 
is  not  available  in  the  NetCDF  files 

ISEC  -  Set  seconds  to  zero 

KM  AX  -  Fixed  at  11-1  =  10  levels  (Extra  level 
needed  for  interpolation  process) 

Current  BoM  files  are  supplied  with  lowest  1 1 
vertical  levels  (up  to  approx  1km  above  the 
surface) 

geopjit  (m) 

topog  (m) 

Used  to  calculate  SZ  (m)  -  vertical  height  levels  of 
grid  (terrain  transformed) 

SZ  is  calculated  using  Equation  (1) 

Ivl  (sigma) 

Pressure  at  any  point  =  Ivl  *  sfc _pres  (used 

sfc _pres  (Pa) 

during  interpolation) 

zonal  wind  (m/s) 

U  -  corresponds  to  interpolated  zonal  (east-west) 
winds  (m/s) 

merid_wind 

V  -  corresponds  to  interpolated  meridional  (north- 

(m/s) 

south)  winds  (m/s) 

omega  (Pa/s) 

W  -  corresponds  to  converted  omega  (vertical) 

Vertical  winds  must  be  interpolated  and  then 

winds  (m/s) 

converted  to  m/s 

airjemp  (K) 

TA  -  Interpolated  air  temperature  (K) 

mix_rto  (g/g) 

H  -  Interpolated  humidity  ratio  (g/g) 

abl_ht  (m) 

Zl  -  Atmospheric  boundary  layer  height  (m) 

Zl  and  HFLX  are  2D  variables  and  require  no 

sfc  ttl  heat  fix 
(W/m^j 

HFLX  -  Total  surface  heat  flux  (W/m2) 

interpolation 

The  geopotential  height  (height  above  mean  sea  level)  is  used  along  with  the 
topography  to  determine  heights  above  the  surface.  The  MEDOC  format  requires  a  list 
of  the  vertical  heights  at  the  southwest  corner  of  the  grid  and  stores  this  information  in 
the  variable  SZ.  SZ  is  fixed  for  each  level,  but  follows  the  terrain  elevation  according  to 
the  formula  found  in  the  User's  Guide  [1]  and  shown  below, 

SZ  =  H(z  -  h)/(H  -  h) ,  (1) 

where  H  is  the  extent  of  the  vertical  aspect  of  the  modelling  domain  (taken  to  be  the 
height  of  the  NetCDF  grid),  z  is  the  height  of  the  position  above  sea  level  (geopotential 
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height  in  our  application)  and  h  is  the  terrain  elevation  (given  by  the  topography 
variable  in  the  NetCDF  file).  Figure  2  (below)  shows  a  3-D  view  of  the  situation  over 
example  topography  with  the  all  variables  from  Equation  (1)  marked. 


Figure  2:  MEDOC  terrain  following  grid  levels  over  3-D  terrain 


The  interpolation  of  the  3D  data  (wind,  temperature,  humidity)  is  then  completed 
using  simple  geometry  and  assuming  that  locally  the  variation  in  these  variables  is 
linear.  This  is  illustrated  in  Figure  3  below,  with  the  subscript  k  indicating  the  vertical 
position  on  the  grid  ( k  -1,2, ...,  11). 
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Level  k+1 

_  _ _ _  ,  _  .  _ _ _  .  _ _s 

Increasing 

Height 

r~ 

CD  1 

<  1 

CD_  1 

3T  1 

1 

l 

l 

1 

1 

l 

flk+l 

2k 

Constant  Pressure  Levels 

i 

«k 

i 

Level  k-1 

Zk-1 

1 

i 

i 

flk-1 

Figure  3:  Linear  interpolation  descriptive  diagram 


Let: 

o  Zk  be  the  geopotential  height  corresponding  to  vertical  level  k, 
o  Ok  be  the  geopotential  height  of  the  original  pressure  level  k  and 
o  x( )  be  a  function  that  returns  3-D  data  (eg.  wind)  at  a  given  position. 

Assuming  x( )  varies  linearly  with  height  implies  the  relation. 


Zk  ~°k 

°k+ 1  —  a 


x(zt)-x(at) 

x(«*+i)“x(fl*) 


x(z*)  =  (x(flt+1)-x(fl*)] 


Zk  ak 
Kak+\  ” ak  J 


+  x(a* ) 


(2) 


Equation  (2)  will  be  used  to  do  the  interpolation  of  all  3-D  data  from  NetCDF  sigma 
levels  to  MEDOC  terrain-following  height  levels. 

The  vertical  winds  presented  a  more  difficult  task  as  they  are  supplied  by  the  BoM  in 
units  of  pascal  per  second  (Pa/s).  Converting  to  metres  per  second  (m/s)  requires  a 
factor  with  units  of  metres  per  pascal,  as  illustrated  below, 

JP[m/s]  =  —[m/s]  =  —  [m/Pa]  ^[Pa/s]  =  —  [m/Pa]  fP[Pa/s] , 
dt  dp  dt  dp 
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where  W,  t  and  p  represent  the  vertical  wind  velocity,  time  and  pressure  respectively, 
with  units  given  in  brackets.  Hence  an  appropriate  conversion  factor  can  be  calculated 
with  units  of  pascal  per  metre  by  evaluating  the  rate  of  change  of  pressure  p  with 
respect  to  height  z  (metres)  at  each  point  on  the  grid.  In  the  meteorology  text  by  Stull 
[6],  this  rate  of  change  is  explicitly  expressed  in  the  form  of  the  hydrostatic  equation,  as 
shown  below. 


dp 

dz 


=  ~gp / 


(3) 


where  g  is  the  acceleration  due  to  gravity  and  p  is  the  density  in  kg/m3.  As  we  are 
dealing  with  only  the  first  kilometre  or  so  above  the  surface  in  this  application,  g  can  be 
assumed  to  be  constant  and  equal  to  9.8  m/s2.  To  express  p  in  terms  of  known 
quantities,  we  use  Boyle's  Law  for  a  perfect  gas,  which  states, 

P  =  RpT ,  (4) 


where  T  represents  the  temperature  (in  units  of  Kelvin)  and  R  is  a  constant  with  value 
of  2.8703  J  /K  kg  when  p  is  in  units  of  millibar.  Combining  Equations  (3)  and  (4)  gives 
the  relation. 


dz  RT 


(5) 


Note  that  the  pressure  data  are  in  units  of  pascals  and  not  millibar  in  the  supplied 
NetCDF  files  so  Equation  (5)  in  our  case  will  be  modified  to, 

dp  g 

—  = - - —  p .  (6) 

dz  100 RT  V  ' 

Hence  the  reciprocal  of  Equation  (6)  is  in  units  of  m/Pa  and  can  be  used  to  convert  the 
vertical  winds  from  Pa/s  to  m/ s  as  required.  More  details  on  the  variation  of  pressure 
with  height  can  be  found  in  most  meteorology  texts,  such  as  that  by  Stull  [6], 

3.4  Reformatting  Formulae 

The  interpolative  formulae  that  the  program  will  use  can  now  be  assembled,  as  shown 
in  Table  6.  Note  that  subscripts  have  been  added  to  most  variables  to  indicate  their 
positions  on  the  3-D  grid,  where  i  indicates  the  east-west  direction,  j  the  north-south 
direction  and  k  the  vertical  direction. 
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Table  6:  NetCDF  to  MEDOC  Iterative  Interpolation  Formulae 


NETCDF  TO  MEDOC  ITERATIVE  FORMULAE 

Variable  Definitions 

NetCDF  Variables 

Ivl 

=  lvlk  where  i  =  1 ,  2 . IMAX 

geop_ht 

=  gijk  j  =  1,  2 . JMAX 

topog 

=  topij  k=  1,  2,  ....  KMAX 

sfc_pres 

=  SPij 

zonal_wind 

=  Uijk 

merid_wind 

=  vijk 

air_temp 

=  tjjk 

omega 

=  Oijk 

mix_rto 

=  mrijk 

abl_ht 

=  hgij 

sfc_ttl_heat_flx 

=  hfij 

Proqram  Variables 

Pijk 

=  Atmospheric  pressure  (Pa) 

TTHyk 

=  Terrain  transformed  height  (m) 

IOijk 

=  Interpolated  vertical  wind  (Pa/s) 

Pfijk 

=  Factor  for  converting  vertical  winds  (m/Pa) 

MEDOC  Variables 

SZk 

=  Vertical  levels  (m) 

Uijk 

=  U  winds  (m/s) 

Vijk 

=  V  winds  (m/s) 

Wijk 

=  W  winds  (m/s) 

H 

> 

‘tt 

=  Absolute  temperature  (K) 

Hp 

=  Humidity  ratio  (g/g) 

TOPOij 

=  Terrain  elevation  (m) 

HFLXy 

=  Total  surface  heat  flux  (W/m2) 

Zlii 

=  Atmospheric  boundary  layer  height  (m) 

DSTO-TN-0595 


Iterative  Formulae 

#  Calculate  height  of  each  vertical  grid  level,  using  Equation  (1 )  # 
for  (k  from  1  to  KMAX)  do 

SZk  =  9hkmax  *  (9iik —  topn)  /  (9hkmax  ~  topn) 
end  for  loop 

#  Calculate  Pressures  and  Terrain  changes  at  each  point  # 
for  ( i  from  1  to  I  MAX)  do 

for  ( j  from  1  to  JMAX)  do 
for  (k  from  1  to  KMAX)  do 

#  Calculate  Pressures  from  sigma  levels  and  surface  pressures  # 

Pijk  =  lvlk  *  SPij 

#  Calculate  Terrain  Transformed  Heights  (TTH)  (essentially  the  geopotential  at  each  point  on 
the  MEDOC  grid)  for  each  position  (using  geopotential  at  corner  as  a  reference  height)  # 

TTHijk  =  gnk  +  topij  -  topn 
end  for  loop 
end  for  loop 
end  for  loop 

#  Start  new  loop  to  interpolate  winds,  temperature  and  humidity  # 
for  (i  from  1  to  IMAX)  do 

for  ( j  from  1  to  JMAX)  do 
for  (k  from  1  to  KMAX)  do 

#  Interpolate  zonal  winds  to  constant  height  level  using  Equation  (2)  # 

Ujjk  —  [Ujjk+i  -  Ujjk]  { (TTHjjk  -  9ijk)  /  (9ijk+i  -  9ijk) }  Ugk 

#  Interpolate  meridional  winds  to  constant  height  level  using  Equation  (2)  # 

Vyk  —  [Vjjk+i  ”  Vjjk]  {  (TTHjjk  ”  9ijk)  I  (9ijk+1  ■  9ijk)  }  Vjjk 

#  Interpolate  air  temperature  to  constant  height  level  using  Equation  (2)  # 

TAjjk  —  [tjjk+i  "  tjjk]  {  (TTHjjk  ”  9ijk)  /  (9ijk+1  “  9ijk)  }  fijk 

#  Interpolate  humidity  ratio  to  constant  height  level  using  Equation  (2)  # 

Hjjk  =  [mrjjk+i  -  mnjk]  { (TTHijk  -  gijk)  /  (g„k*i  -  g«0 }  +  mrljk 

#  Interpolate  omega  (vertical  winds)  to  constant  height  level  using  Equation  (2)  # 

lOjjk  =  [Ojjk+i  -  Ojjk]  { (TTHp  ■  g^jk)  /  (9ijk+i  -  gijk) }  +  Ojjk 

#  Convert  omega  from  Pa/s  to  m/s  by  dividing  by  factor  with  units  Pa/m  (use  Equation  (6))  # 

pfijk  =  -0.034143  *  Pjjk  /  TAjjk 

Wjjk  —  IOjjk  /  pfijk 
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#  Map  2D  variables  to  MEDOC  equivalents  # 

TOPOy  =  topg 

HFLXjj  =  hfjj 
Zly  =  bhy 
end  for  loop 
end  for  loop 
end  for  loop 

Note  that  the  interpolation  process  is  linear  in  all  variables  and  takes  into  account  the 
changing  topography. 


4.  AWSMSTR  Program 

AWSMSTR .  EXE  is  a  program  which  sits  in  the  background  on  32-bit  Microsoft  Windows 
Systems  (Windows  95  or  later).  At  specified  intervals  it  runs  a  batch  file  called 
AWSLIST.BAT  which  then  invokes  an  FTP  (File  Transfer  Protocol)  script  called 
AWSLIST .  SCP  which  uses  the  FTP  program  to  download  over  the  Internet  a  directory 
listing  from  a  remote  FTP  server  which  is  then  saved  in  a  file  named  AWS .  LST.  The 
program  then  constructs  a  directory  listing  file  named  LOC .  LST  for  the  directory  in 
which  it  resides. 

Just  prior  to  running  the  batch  file,  AWSMSTR  checks  for  the  presence  of  a  file 
AWSMSTR.  FLG.  If  this  file  (which  was  created  by  AWSMSTR  when  it  was  first  started) 
no  longer  exists  then  AWSMSTR  will  cease  execution  and  remove  itself  from  memory. 
The  deletion  of  this  'flag'  file  is  thus  the  method  provided  for  closing  the  AWSMSTR 
program.  Any  messages  or  exception  output  created  by  AWSMSTR  are  placed  in  the  file 
AWSERROR.LOG. 

The  program  then  searches  the  two  directory  listings  for  files  which  will  be  of  interest 
because  their  names  match  templates  specified  in  the  file  AWSFILE .  TFN  (see  Appendix 
D  for  a  description  of  this  file),  creates  an  internal  list  of  files  which  are  present  on  the 
FTP  server  but  not  in  the  local  directory,  and  deletes  any  matching  files  which  are 
present  in  the  local  directory  but  not  present  on  the  FTP  server.  It  then  uses  the 
internal  list  of  files  in  conjunction  with  the  header  file  AWSGETSC .  HDR  to  construct  an 
FTP  script  called  AWSGET .  SCP.  AWSMSTR  then  runs  a  batch  file  called  AWSGET .  BAT 
which  uses  the  FTP  script  just  constructed  to  connect  to  the  FTP  server  and  download 
the  files  of  interest  which  are  present  on  the  FTP  server  but  not  in  the  local  directory. 
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Having  downloaded  the  required  files,  AWSMSTR  then  submits  the  downloaded  files  to 
a  translator  program,  one  at  a  time  and  then  becomes  idle  until  the  next  specified 
download  interval. 

The  AWSMSTR  program  must  be  invoked  using  a  command  line  of  the  form: 

AWSMSTR  <interval  minutes>  <keyword>  <translator  program>  coutput  directory^ 

eg.  AWSMSTR  30  melbourne  .\BOM2HPAC.EXE  . \OUTPUT 

The  translator  program  written  for  this  exercise  is  called  BOM2HPAC.EXE.  When  run, 
this  program  will  translate  the  file  specified  as  the  first  argument  on  its  command  line 
and  place  the  output,  suitably  named,  into  the  directory  specified  as  the  second 
argument  on  its  command  line.  When  called  by  AWSMSTR,  BOM2HPAC  will  be  supplied 
with  the  correct  file  name  and  the  output  directory  which  was  specified  on  the 
command  line  for  AWSMSTR. 

BOM2HPAC  will  use  correct  algorithms  to  translate  and  rename  several  different  types 
of  input  files  using  filename  template  data  supplied  in  the  file  BOM2HPAC.TFN  as 
specified  in  Appendix  E. 


5.  Downloading  Process 

The  downloading  process  is  quite  simple  but  takes  a  few  moments  to  set  up.  The 
procedure  for  setting  up  the  program  is  detailed  in  Appendix  C.  Once  set  up,  there  are 
two  shortcuts  on  the  desktop,  one  for  downloading  observational  data  and  one  for 
downloading  the  gridded  forecast  data  (note  that  at  this  stage  it  has  not  been  possible 
to  have  a  single  executable  that  downloads  both  observational  and  forecast  data 
simultaneously  as  the  data  are  not  available  in  a  single  directory  on  the  BoM  ftp 
server).  Each  of  these  will  run  in  the  background,  with  a  command  window  popping 
up  to  indicate  that  a  connection  to  the  BoM  ftp  server  is  being  made  and  new  files 
downloaded.  There  are  three  types  of  files  that  are  output  ready  for  use  in  HPAC, 
surface  observations  in  METAR  format  ( .  sao  extension),  upper-air  observations  in 
RAOBS  format  (.  wmo  extension)  and  gridded  MEDOC  data  (.  fmt  extension).  Exiting 
the  AWSMSTR  program  is  a  simple  process  of  deleting  the  two  awsmstr .  fig  files  that 
are  contained  in  the  downloading  directories. 

When  importing  meteorological  data  the  AWSMSTR  program  appends  the  date  and  time 
of  the  data  to  the  filename  (see  Appendix  D),  with  all  times  recorded  in  UTC  (Universal 
Time  Coordinated).  This  is  especially  important  when  deciding  which  files  to  import 
for  an  HPAC  modelling  run. 

The  MEDOC  files  are  then  ready  to  import  into  HPAC  via  the  Weather  Wizard's  Use 
Existing  HPAC  Files  As  Is  option.  A  single  MEDOC  file  is  output  for  each  36  hour 
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forecast,  with  sizes  nearly  2.5  times  larger  than  the  total  size  of  the  forecast  files 
downloaded.  This  increase  in  size  is  due  to  the  less  efficient  (compared  to  NetCDF) 
way  that  the  MEDOC  format  is  structured.  The  largest  MEDOC  files  that  are  produced 
are  from  the  Sydney  region  and  are  nearly  42  MB  in  size.  HP  AC  has  little  trouble 
coping  with  file  sizes  like  this,  with  test-run  times  still  very  low. 
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Appendix  A:  Surface  Observation  Examples 


A.l.  [metar2Data].axf  type  example 


[metar2Data] 

ID  num,  ID  name  [6], 

Date,  Time,  Lat,  Lon 

,  Wdir,  Wspd, 

T_DB, 

DP,  QNH, 

RF9am,  RFlOm,  Vis, 

AVis,  Gust,  Wxllnt,  WxlDsc,  WxlWxl, 

WxlWx2 

,  WxlWx3, 

Wx2Int,  Wx2Dsc,  Wx2Wxl,  Wx2Wx2, 

Wx2Wx3, 

CldlAmt,  CldlTyp, 

CldlBase, 

Cld2Amt ,  Cld2Typ, 

Cld2Base,  Cld3Amt, 

Cld3Typ,  Cld3Base, 

Cld4Amt , 

Cld4Typ,  Cld4Base, 

CeillAmt 

,  CeillBase,  Ceil2Amt,  Ceil2Base, 

Ceil3Amt , 

Ceil3Base 

95204,  "WSR  ",  20031116, 

0600, 

-17.90, 

122.31,  260, 

13.0, 

32.5,  - 

9999.0,  -9999.0, 

o 

o 

0.0, 

-9999, 

-9999,  20.0, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999, 

-9999, 

-9999, 

-9999,  -9999, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999, 

-9999, 

-9999, 

-9999,  -9999, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999,  ■ 

-9999, 

-9999 

94430,  "YMEK  ",  20031116, 

0546, 

-26.61, 

118.54,  030 

,  17.0 

,  30.1, 

11.3,  1004.6,  0.0 

o 

o 

10000 

,  10000, 

29.0,  -9999, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999, 

-9999, 

-9999, 

-9999,  1,  7, 

600 

,  3,  8, 

2400,  3,  9,  2100,  -9999 

,  -9999,  -9999,  11,  2490, 

-9999, 

-9999,  - 

9999,  -9999 

94317,  " YNWN  ",  20031116, 

0539, 

-23.42, 

,  119.80,  350 

o 

VD 

i — 1 

,  34.2, 

10.1,  1007.8,  0.0 

,  o.o, 

-9999 

,  -9999, 

26.0,  -9999, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999, 

-9999, 

-9999, 

-9999,  -9999, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999, 

-9999, 

-9999, 

-9999,  -9999, 

-9999, 

-9999,  - 

9999,  -9999,  -9999, 

-9999,  ■ 

-9999 

A.2.  HPAC's  METAR  format  for  surface  observations  example 


METAR 

YHLC 

161230Z 

15009KT 

//// 

24/20 

Q1012 

METAR 

YPKU 

161230Z 

01008KT 

//// 

29/21 

Q1010 

METAR 

YARG 

161230Z 

34007KT 

//// 

32/19 

Q1011 

METAR 

YBRM 

161230Z 

28011KT 

nn 

28/22 

Q1011 

METAR 

YPPD 

161230Z 

28012KT 

nn 

25/15 

Q1011 

METAR 

YPKA 

161230Z 

29014KT 

nn 

25/16 

Q1011 

METAR 

YPLM 

161230Z 

28009KT 

nn 

23/16 

Q1011 

METAR 

YCAR 

161230Z 

32009KT 

n  /  / 

22/17 

Q1009 

METAR 

YMEK 

161230Z 

22006KT 

//// 

22/12 

Q1009 

METAR 

YGEL 

161230Z 

31004KT 

//// 

19/17 

Q1009 

METAR 

YPPH 

161230Z 

21010KT 

//// 

20/15 

Q1007 

METAR 

DOPY 

161230Z 

18014KT 

//// 

///// 

Q//// 

METAR 

YPJT 

161230Z 

22012KT 

//// 

19/15 

Q1007 

METAR 

YRTI 

161230Z 

19020KT 

//// 

18/15 

Q1006 

METAR 

OCEA 

161230Z 

20017KT 

//// 

///// 

Q//// 

METAR 

SWAN 

161230Z 

20013KT 

//// 

19/16 

Q//// 

METAR 

PERT 

161230Z 

21009KT 

//// 

20/15 

Q1007 

METAR 

YESP 

161230Z 

06005KT 

//// 

17/13 

Q1011 

METAR 

YABA 

161230Z 

07009KT 

//// 

15/14 

Q1007 

METAR 

YEST 

161230Z 

07005KT 

//// 

17/14 

Q1011 

METAR 

ECLT 

161230Z 

12007KT 

//// 

16/14 

Q1011 

- 

21 


DSTO-TN-0595 


22 


DSTO-TN-0595 


Appendix  B:  Gridded  Forecast  Data  -  MEDOC 

Example 


FFFFFFFF 

TRNSGRIB 

11 

11 

2003 

12 

0 

0 

11 

11 

2003 

12 

0 

0 

11 

19 

10 

0 

5 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

10.1713 

21.9336 

48.0274 

105.5031 

212.0182 

319.6505 

428.3995 

649.4742 

875.9681 

1108.6001 

0.1250 

0.1250 

-999999.0000  -999999.0000 

-43.2500 

146.7500 

0.0000 

0.0000 

0.0000 

0.0000 

0.0000 

U  V 

w 

TA  H 

M/S  M/S 

M/S 

KELVIN  GM/GM 

TOPO  METERS 

HFLX 

W/m**2  ZI 

M 

14.1074 

11.7645 

9.4437 

8.8821 

9.4905 

10.1203 

10.7202 

11.3682 

11.9817 

12.5280 

13.2309 

9.2971 

9.4522 

9.3462 

9.2524 

8.4669 

8.1637 

8.1594 

9.4280 

11.4099 

12.5430 

12.5580 

8.3281 

8.1465 

8.6042 

9.8755 

8.9849 

7.1762 

5.7191 

7.1310 

9.6505 

11.3072 

11.4873 

8.5472 

7.9655 

8.4571 

9.2701 

8.4498 

7.0408 

4.8109 

5.9415 

8.3236 

9.9089 

10.3761 

6.9883 

6.7481 

6.9886 

7.0376 

6.9724 

7.1990 

6.6003 

7.2729 

8.5935 

9.3220 

9.8514 

6.0430 

5.9069 

6.0163 

6.0677 

6.2970 

6.8710 

7.6036 

8.4942 

8.8278 

8.4706 

9.1721 

5.5502 

5.5143 

5.6348 

5.8513 

6.0079 

6.6891 

7.4764 

8.2701 

8.9252 

7.8046 

8.1529 

5.4278 

5.3123 

5.6584 

6.4639 

6.4542 

6.0617 

6.3068 

7.0031 

7.7835 

7.5213 

6.9956 

5.7464 

5.4900 

6.6537 

8.6462 

6.9971 

4.6859 

4.2701 

5.4281 

7.0133 

6.9584 

6.0480 

6.3856 

6.6591 

8.1983 

9.3517 

6.6847 

3.7711 

3.4776 

5.1668 

7.7495 

FFFFFFFF 

TRNSGRIB 

• 

11 

11 

2003 

15 

0 

0 

11 

11 

2003 

15 

0 

0 

11 

19 

10 

0 

5 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

10.1266 

21.8309 

47.7782 

104.8865 

210.6732 

317.5629 

425.5577 

644.8875 

869.2156 

1099.5909 

0.1250 

0.1250 

-999999.0000  -999999.0000 

-43.2500 

146.7500 

0.0000 

0.0000 

0.0000 

0.0000 

0.0000 

U  V 

W 

TA  H 

M/S  M/S 

M/S 

KELVIN  GM/GM 

TOPO  METERS 

HFLX 

W/m**2  ZI 

M 

2.0468 

1.3312 

2.3943 

3.4582 

3.2527 

5.6552 

7.3278 

8.8878 

9.8600 

11.1821 

8.5576 

2.8381 
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Appendix  C:  AWSMSTR.exe  Set-up  Procedure 

Step  1  -  Create  observational  and  NWP  directories  in  HPAC  \rawmet  directory. 

Step  2  -  Unzip  obs .  zip  into  observation  folder,  nwp .  zip  into  NWP  folder 
Step  3  -  Create  shortcut  for  downloading  observational  data 
Step  4  -  Create  shortcut  for  downloading  forecast  data,  and  edit  to  include  the 
gridded  area  of  interest. 

Step  5  -  Adjusting  AWSMSTR.exe  for  non-Defence  use 
Step  6  -  Additional  notes  and  technical  support 

□  Step  1  -  Create  Directories 

•  HPAC  will  by  default  look  in  a  directory  called  \rawmet  when  importing 
both  observational  and  forecast  weather  data.  It  makes  sense  then  to  set  up 
the  downloading  programs  from  this  directory. 

•  Find  the  \rawmet  directory,  usually  at 
C : \HPAC4\client\shared\rawmet 

•  Add  two  new  directories,  \obsdwnld  and  \nwpdwnld 

•  The  observational  weather  data  will  be  saved  in  \obsdwnld,  and  the 
forecast  data  will  be  put  in  \nwpdwnld 

□  Step  2  -  Unzip  Program  Files 

•  Using  WinZip  or  another  similar  extraction  program,  extract  the  contents  of 
obs . zip  into  C : \HPAC4\client\shared\rawmet\obsdwnld 

•  Extract  nwp.  zip  into  C:  \HPAC4 \client\shared\rawmet\nwpdwnld 

□  Step  3  -  Create  Observations  Downloading  Shortcut 

•  In  \obsdwnld,  find  the  program  AWSMSTR.exe 

•  Right-click  and  select  Create  Shortcut 

•  Drag  this  shortcut  to  the  desktop 

•  Right-click  on  desktop  icon  and  select  Properties 

•  Click  on  the  Shortcut  tab 

•  Locate  the  Target:  editable  field. 

•  This  field  contains  5  pieces  of  important  information  for  the  downloading 
process: 

o  The  name  and  location  of  AWSMSTR .  exe 

o  The  time  interval  (in  minutes)  between  looking  for  updated  files  on 
the  BoM  ftp  server 

o  The  city/cities  for  which  to  download  the  large  forecast  files 
o  The  location  and  name  of  the  translator  program 
o  The  destination  of  converted  files 

•  An  example  is  shown  below  which  downloads  all  observations  every  30 
minutes,  gets  forecast  data  from  Melbourne  (AWSMSTR  requires  the 
keyword,  but  note  that  this  incarnation  of  AWSMSTR  will  download 
observations  only),  and  puts  files  in  the  current  directory. 
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•  C:\HPAC4\client\shared\rawmet\obsdwnld\AWSMSTR.exe  30 
melbourne 

C : \HPAC4\client\shared\rawmet\obsdwnld\BOM2HPAC . exe 
C : \HPAC4\client\shared\rawmet\obsdwnld 

•  Rename  the  desktop  shortcut  "Observations  Download"  or  something 
similar 

•  When  run  the  program  downloads  and  reformats  all  surface  and  upper-air 
observational  data  from  around  Australia  and  saves  files  in  the  \obsdwnld 
directory 

□  Step  4  -  Create  Forecast  Downloading  Shortcut 

•  The  procedure  is  virtually  identical  for  the  forecast  data  download 

•  In  \nwpdwnld,  find  the  program  AWSMSTR .  exe 

•  Right-click  on  select  Create  Shortcut 

•  Drag  this  shortcut  to  the  desktop 

•  Right-click  on  the  desktop  icon  and  select  Properties 

•  Click  on  the  Shortcut  tab 

•  Locate  the  Target:  editable  field 

•  This  field  contains  5  pieces  of  important  information  for  the  downloading 
process: 

o  The  name  and  location  of  AWSMSTR .  exe 

o  The  time  interval  (in  minutes)  between  looking  for  updated  files  on 
the  BoM  ftp  server 

o  The  city/ cities  for  which  to  download  the  large  forecast  files 
o  The  location  and  name  of  the  translator  program 
o  The  destination  of  the  converted  files 

•  An  example  is  shown  below  which  downloads  all  forecast  data  from 
Melbourne  (checking  for  new  data  every  120  minutes),  and  saves  the  files  in 
the  current  directory 

•  C:\HPAC4\client\shared\rawmet\nwpdwnld\AWSMSTR.exe  120 
melbourne 

C : \HPAC4 \ client \shared\rawmet\nwpdwnld\BOM2HPAC . exe 
C : \HPAC4 \client\shared\rawmet\nwpdwnld 

•  Rename  the  desktop  shortcut  "Forecast  Download"  or  something  similar 

•  When  run,  the  program  downloads  and  reformats  all  forecast  data  for  the 
specified  city/cities  and  saves  files  in  the  \nwpdwnld  directory.  Note  that 
the  forecast  data  is  only  updated  every  12  hours,  at  approx  0300  and  1500 
UTC  (1pm  and  lam  AEST) 

□  Step  5  -  Adjust  For  Non-Defence  Use 

•  AWSMSTR  was  designed  primarily  for  use  within  Defence  networks  but  can 
be  easily  adjusted  for  use  on  any  network  by  adjusting  the  contents  of 
awsget.bat,  awslist.scp  and  awsgetsc.hdr  to  reflect  the  ftp  proxy 
appropriate  to  the  end-user.  The  AWSMSTR  software  is  available  to  licensed 
HPAC  users  upon  request  to  the  authors. 
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Step  6  -  Additional  Notes 

•  Also  required  for  use  of  observational  data  is  the  updated  version  of  the 
weather  station  file,  wmo.stn.  Run  a  search  on  the  HPAC  directory  and 
locate  all  instances  of  wmo.stn  (there  should  be  two).  Rename  each  file 
wmo_old.stn  and  copy  the  new  DSTO  supplied  wmo.stn  into  these 
directories. 

•  Due  to  the  non-uniqueness  inherent  in  METAR  files  (only  the  time  and  day 
of  the  month  is  recorded),  it  is  suggested  that  no  more  than  one  month  of 
downloaded  observations  is  kept  in  the  \rawmet  directory,  as  HPAC  may 
fail  to  import  observations  or  incorrectly  assign  observations  to  the  current 
HPAC  project's  month.  Given  the  short-term  nature  of  most  dispersion 
scenarios  this  will  rarely  be  an  issue. 

•  Note  that  at  this  stage  the  BoM  are  unable  to  provide  all  data  in  a  single 
directory  on  their  ftp  server,  hence  the  need  for  two  programs,  one 
downloading  observational  data,  the  other  the  NWP  forecast  data.  It  is 
hoped  that  in  the  future  the  BoM  can  provide  all  data  in  a  single  directory 
so  that  only  one  program  will  be  required  to  download  all  BoM  data. 

•  Possible  keywords  to  specify  the  area  of  interest  for  NWP  data  are: 

o  Melbourne  and  surrounds  melbourne 

o  Sydney  and  surrounds  Sydney 

o  Canberra  region  Canberra 

o  Adelaide  and  surrounds  adelaide 

o  Brisbane  and  Gold  Coast  brisbane 

o  Darwin  region  darwin 

o  Perth  and  surrounds  perth 

o  Hobart  and  Launceston  tasmania 

•  To  download  forecasts  from  more  than  one  region,  simply  separate  as  many 
of  the  keywords  as  required  with  commas.  For  example  to  download  both 
Melbourne  and  Sydney  data,  use  the  keywords  melbourne,  Sydney  in  the 
command  line. 

•  Note  that  a  keyword  (or  more  than  one)  must  be  included  in  the  command 
line.  For  the  observations-only  program  run,  inclusion  of  a  keyword  is 
mandatory  but  does  not  affect  the  files  downloaded,  which  still  contain  all 
observational  data  from  around  Australia. 

•  To  stop  the  programs  from  running,  locate  the  two  awsmstr  .fig  files  (one 
in  each  of  the  \obsdwnld  and  \nwpdwnld  directories)  and  delete. 


If  you  have  any  problems,  contact  Alex  Hill  at  DSTO  via  email 
falexander.hill@dsto.defence.eov.au’),  telephone  03)  9626  8272  or  mobile  0439  463  252 
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Appendix  D:  AWSFILE.TFN  Templates 


The  file-name  templates  specified  in  this  file  are  lines  having  the  form  of  a  'keyword' 
followed  by  an  equals  (=)  character,  followed  by  a  list  of  templates  which  are  enclosed 
in  double-quotes  and  separated  by  commas. 

keyword="template",  "template",  "template",.... 

Templates  corresponding  to  the  keyword  'global'  will  always  be  used  in  addition  to  the 
templates  corresponding  to  one  other  keyword  which  is  specified  on  the  AWSMSTR 
command  line.  A  filename  need  match  only  one  of  the  specified  templates  in  order  to 
be  accepted  as  a  match.  The  current  AWSFILE .  TFN  file  is  shown  in  Figure  Dl. 


global="* . txt",  . axf " 

melbourne="IDY04287_>4Y>M>D>h_>3d. nc" 
sydney="IDY04288_>4Y>M>D>h_>3d. nc" 
canberra="IDY04289_>4Y>M>D>h_>3d. nc" 
adelaide="IDY04290_>4Y>M>D>h_>3d. nc" 
brisbane="IDY04291_>4Y>M>D>h_>3d. nc" 
darwin="IDY04292_>4Y>M>D>h_>3d. nc" 
perth="IDY04293_>4Y>M>D>h_>3d . nc" 
tasmania="IDY04299  >4Y>M>D>h  >3d.nc" 


Figure  Dl:  Current  AWSFILE.TFN 


Wildcards  accepted  in  the  file  name  templates  are: 

'*'  for  zero  or  more  arbitrary  characters; 

'?'  for  any  one  character. 

Unlike  the  MS-DOS  wildcard  match,  is  correctly  handled  even  if  it  isn't  at  the  end  of 
the  template.  Note  that  the  period  Y  is  not  a  special  character.  The  wildcard  handling 
code  was  originally  written  by  Jeff  Damens  of  Columbia  University's  Center  for 
Computing  Activities  and  is  taken  from  the  source  code  for  C-Kermit  version  4C  [7], 

File  name  templates  can  also  include  text-substitution  macros  as  specified  below.  The 
first  character  of  all  macros  is  a  character  which  is  not  permitted  in  filenames  and  is  not 
a  wildcard  -  viz:  the  ">"  character.  The  next  character  may  be  a  decimal  digit  which 
specifies  the  size  of  a  character  string  which  comprises  the  macro  or,  if  absent  (with  the 
exception  of  the  d  macro),  the  macro  is  only  two  characters  long  and  the  matching 
string  only  two  characters  long. 

>4Y  A  4-digit  year  eg.  2003 

>2Y  or 

>Y  A  2-digit  year  specifying  the  number  of  years  since  the  last  century 
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eg.03  ==  2003  or  1903  or  ... 

>2M  or 

>M  A  2-digit  month  of  the  year  (1  - 12)  eg.  07  ==  July 
>2D  or 

>D  A  2-digit  day  of  the  month  (1-31)  eg.  09 
>2h  or 

>h  A  2-digit  hour  of  the  day  (0  -  23)  eg  06 
>2m  or 

>m  A  2  -  digit  minute  of  the  hour  (0  -  59)  eg.  05 
>2s  or 

>s  A  2  -  digit  second  of  the  minute  (0  -  59)  eg.  02 

>ld  or  >2d  or  >3d  or  >4d  or  ...  a  decimal  integer  having  the  specified  number 
of  digits  OR 

>d  A  decimal  integer  having  an  unspecified  number  of  digits  -  the  string  to  be 
matched  is  parsed  until  the  end  or  until  the  first  non-digit  is  encountered. 
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Appendix  E:  BOM2HPAC.TFN  Templates 

The  file  name  templates  specified  in  this  file  have  the  form: 

keyword=[Input  Filename  Template  List],[Output  Filename  Template  List], 

where  any  template  list  has  each  template  enclosed  in  double-quotes  and  separated  by 
commas  as  in  Appendix  D.  The  BOM2HPAC  program  searches  each  of  the  input 
template  lists  until  it  finds  a  match  for  the  file  specified  on  the  command  line.  It  then 
uses  the  keyword  located  on  the  same  line  to  determine  which  translation  algorithm  to 
use.  Time,  date  and  numeric  information  extracted  from  the  construction  of  the  input 
filename  macros  are  used  in  conjunction  with  the  output  filename  template  list  located 
on  the  same  line  as  the  keyword  to  determine  how  to  name  the  output  file. 

The  current  B0M2HPAC  .  TFN  file  is  shown  in  Figure  El. 


UPPER_AIR_OBS= [ "IDY07033* . txt" ] , [ "IDY07033* . wmo" ] 

SURFACE_OBS= [ "IDY03100* . axf " ] , [ "IDY03100* . sao" ] 

PREP  36HR= [ "IDY042??  >4Y>M>D>h  >d . nc" ],[" ????????  >4Y>M>D>h . fmt" ] 

Figure  El:  Current  BOM2HPAC.TFN  File  Name  Templates  File 

Wildcards  and  macros  which  may  be  included  in  this  file  are  as  specified  in 
Appendix  D. 
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